<html>
<head><meta charset="utf-8"><title>Initial benchmarks · t-compiler/wg-parallel-rustc · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/index.html">t-compiler/wg-parallel-rustc</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html">Initial benchmarks</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="183724707"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183724707" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183724707">(Dec 18 2019 at 06:50)</a>:</h4>
<p>I posted some initial benchmarks at <a href="https://internals.rust-lang.org/t/plan-to-test-parallel-rustc/11487/15" target="_blank" title="https://internals.rust-lang.org/t/plan-to-test-parallel-rustc/11487/15">https://internals.rust-lang.org/t/plan-to-test-parallel-rustc/11487/15</a> and <a href="https://internals.rust-lang.org/t/plan-to-test-parallel-rustc/11487/16" target="_blank" title="https://internals.rust-lang.org/t/plan-to-test-parallel-rustc/11487/16">https://internals.rust-lang.org/t/plan-to-test-parallel-rustc/11487/16</a> , and they look good!</p>



<a name="183726454"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183726454" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183726454">(Dec 18 2019 at 07:35)</a>:</h4>
<p>The difference in output sizes are surprising. Do these also happen with <code>-Z threads=1</code>?</p>



<a name="183726534"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183726534" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183726534">(Dec 18 2019 at 07:37)</a>:</h4>
<p>Also you didn't mention how many cores your CPU has. You may want to specify a higher thread count to rustc (it defaults to a maximum of 4).</p>



<a name="183778772"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183778772" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183778772">(Dec 18 2019 at 18:50)</a>:</h4>
<p>So simon's question on internals about "what's still serial" led me to investigate a little</p>



<a name="183778789"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183778789" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183778789">(Dec 18 2019 at 18:50)</a>:</h4>
<p>Using <code>-Z self-profile</code> it's actually really apparent what's still serial (yay!)</p>



<a name="183778824"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183778824" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183778824">(Dec 18 2019 at 18:51)</a>:</h4>
<p>I compiled Cargo itself with <code>-Z self-profile</code> with our parallel nightly, and I'm actually pretty surprised by the results</p>



<a name="183778873"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183778873" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183778873">(Dec 18 2019 at 18:51)</a>:</h4>
<p>this is the overall picture: <a href="/user_uploads/4715/X8afbWRlKa38KrufurG36y3Z/Screen-Shot-2019-12-18-at-12.51.36-PM.png" target="_blank" title="Screen-Shot-2019-12-18-at-12.51.36-PM.png">Screen-Shot-2019-12-18-at-12.51.36-PM.png</a></p>
<div class="message_inline_image"><a href="/user_uploads/4715/X8afbWRlKa38KrufurG36y3Z/Screen-Shot-2019-12-18-at-12.51.36-PM.png" target="_blank" title="Screen-Shot-2019-12-18-at-12.51.36-PM.png"><img src="/user_uploads/4715/X8afbWRlKa38KrufurG36y3Z/Screen-Shot-2019-12-18-at-12.51.36-PM.png"></a></div>



<a name="183778938"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183778938" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183778938">(Dec 18 2019 at 18:52)</a>:</h4>
<p>type checking is suprisingly not really parallel at all <a href="/user_uploads/4715/kNUbgaJK8h030X92ZVPpGg5i/Screen-Shot-2019-12-18-at-12.51.59-PM.png" target="_blank" title="Screen-Shot-2019-12-18-at-12.51.59-PM.png">Screen-Shot-2019-12-18-at-12.51.59-PM.png</a></p>
<div class="message_inline_image"><a href="/user_uploads/4715/kNUbgaJK8h030X92ZVPpGg5i/Screen-Shot-2019-12-18-at-12.51.59-PM.png" target="_blank" title="Screen-Shot-2019-12-18-at-12.51.59-PM.png"><img src="/user_uploads/4715/kNUbgaJK8h030X92ZVPpGg5i/Screen-Shot-2019-12-18-at-12.51.59-PM.png"></a></div>



<a name="183778988"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183778988" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183778988">(Dec 18 2019 at 18:53)</a>:</h4>
<p>and partitioning is only really barely parallel -- <a href="/user_uploads/4715/kerPyXBj_u_jIsnMnl9j2cv_/Screen-Shot-2019-12-18-at-12.52.35-PM.png" target="_blank" title="Screen-Shot-2019-12-18-at-12.52.35-PM.png">Screen-Shot-2019-12-18-at-12.52.35-PM.png</a></p>
<div class="message_inline_image"><a href="/user_uploads/4715/kerPyXBj_u_jIsnMnl9j2cv_/Screen-Shot-2019-12-18-at-12.52.35-PM.png" target="_blank" title="Screen-Shot-2019-12-18-at-12.52.35-PM.png"><img src="/user_uploads/4715/kerPyXBj_u_jIsnMnl9j2cv_/Screen-Shot-2019-12-18-at-12.52.35-PM.png"></a></div>



<a name="183779008"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183779008" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183779008">(Dec 18 2019 at 18:53)</a>:</h4>
<p>is a lot of this work sort of just not inherently that parallel? or are things not getting stolen?</p>



<a name="183779082"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183779082" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183779082">(Dec 18 2019 at 18:54)</a>:</h4>
<p>this was just a single crate build via <code>cargo +nightly-2019-12-18 rustc --lib -- -Z self-profile</code> after a full build, so it was only building the one library crate</p>



<a name="183876635"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183876635" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183876635">(Dec 19 2019 at 19:24)</a>:</h4>
<p>I'm not sure I know how to interpret those graphs</p>



<a name="183876644"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183876644" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183876644">(Dec 19 2019 at 19:24)</a>:</h4>
<p>what do "parallel things" look like?</p>



<a name="183876993"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183876993" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183876993">(Dec 19 2019 at 19:28)</a>:</h4>
<p>on the left there are thread ids</p>



<a name="183877014"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183877014" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183877014">(Dec 19 2019 at 19:28)</a>:</h4>
<p>and I think we're only parallel if we have multiple thread ids with simultaneous bars</p>



<a name="183878299"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183878299" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183878299">(Dec 19 2019 at 19:43)</a>:</h4>
<p>Yeah, that's correct. As a concrete example, if you look in the first image "Screen-Shot-2019-12-18-at-12.51.36-Pm.png", there are two purple bars on Thread 3 (see left hand gutter) around 1350 - 1380ms (see top bar for timeline) and a purple bar and a green bar at the same time on Thread 4. The two overlapping purple bars and the overlapping purple and green bars mean threads 3 &amp; 4 were doing work at the same time.</p>



<a name="183896208"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183896208" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183896208">(Dec 19 2019 at 23:37)</a>:</h4>
<p><span class="user-mention" data-user-id="116015">@Alex Crichton</span> Only wf checking and item-bodies checking is parallel within type_check_crate.</p>
<p><span class="user-mention" data-user-id="125250">@Wesley Wiser</span> Why doesn't <code>-Z self-profile</code> include <code>time</code> calls?</p>



<a name="183900115"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183900115" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183900115">(Dec 20 2019 at 00:45)</a>:</h4>
<p><span class="user-mention" data-user-id="116466">@Zoxc</span> No one's asked for it before <span aria-label="slight smile" class="emoji emoji-1f642" role="img" title="slight smile">:slight_smile:</span></p>



<a name="183902307"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/183902307" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#183902307">(Dec 20 2019 at 01:38)</a>:</h4>
<p>I'm sure I mentioned it on some PR, since there's locations where there's both a self-profile timer and <code>time</code> call, which seems a bit redundant.</p>



<a name="184227953"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial%20benchmarks/near/184227953" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/187679-t-compiler/wg-parallel-rustc/topic/Initial.20benchmarks.html#184227953">(Dec 25 2019 at 19:04)</a>:</h4>
<p><span class="user-mention" data-user-id="239881">@Josh Triplett</span> Maybe you could do some benchmarks while bumping the shard count (You can adjust it in <code>sharded.rs</code>). It's only been tuned for my 8-core R7 1700 =P</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>